WHAT IS CLAIMED IS: 



1. A method of measuring the surface motion of each portion of an 
object surface from a sequence of unit images of the object including first 
and second unit images, comprising the steps of: 

determining a plurality of measurement points in an image of said 
object of a first unit image, said plurality of measurement points having a 
hierarchical structure of a plurahty of levels, with the higher level layer 
including measurement points larger in number than the lower level layer; 

determining the points in said object image of said second unit image, 
corresponding to the measurement points of said first unit image of the 
lowest level of the hierarchical structure of said plurality levels; and 

repeating, starting from a level immediately upper than the lowest 
level to the highest level, the determination of the points corresponding to 
the measurement points of the first unit image at each level in the image of 
said object of said second unit image, based on the points corresponding to 
said measurement points of said first unit image at an immediately lower 
level of each level in the image of sedd object of said second unit image. 

2. The method of measuring surface motion according to claim 1, 
wherein said step of determining measurement points includes the steps of 

determining a plurality of reference points of said object image of 
said first unit image, and 

determining, based on the plurahty of reference points, arrangement 
of the measurement points having a predetermined distribution. 

3. The method of measuring surface motion according to claim 2, 
wherein said step of determining the arrangement of the measurement 
points includes the step of mapping a plurahty of points arranged at 
prescribed positions on a pre-selected prescribed three-dimensional plane in 
accordance with the three-dimensional shape of the object surface, to an 
image plane of said image, using the plurahty of reference points as the 
reference. 
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4. The method of measuring surface motion according to claim 3, 
further comprising the step of 

determining the three-dimensionsd position of said object by 
reverse-projecting, onto a prescribed three-dimensional plane, the points of 
i the image of said object in the second unit image corresponding to said 
measurement points of said first unit image. 

5. The method of measuring surface motion according to claim 3, 
wherein said step of determining the points corresponding to the 
measurement points includes the steps of 

obtaining subband images by wavelet transform, with a plurality of 
J 5 resolutions of the same number as said plurahty of levels, the object images 
of said first and second unit images, the plurahty of resolutions being in 
correspondence with the plurality of levels and resolution corresponding to a 
lower level being selected to be lower; 

determining the correspondence between the corresponding subband 
10 images of the lowest level and the measurement points of said lowest level, 
for said first unit image; 

determining image segments having a prescribed relation with each 
of the measurement points of said lowest level of said subband images of 
said lowest level obtained for said first unit image; and 
15 determining the positions of points in said second unit image, 

corresponding to each of the measurement points of said lowest level in said 
first unit image, by determining an image segment of said subband image of 
said lowest level obtained for said second unit image that represents the 
highest correlation with the texture of each said image segment. 

6. The method of measuring surface motion according to claim 5, 
wherein said step of determining the points corresponding to said 
measurement points further includes the step of estimating the position of 
each of the measurement points at the level immediately higher than said 

5 lowest level by interpolating position of the point corresponding to each of 
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the measurement points at the lowest level. 

7. The method of measuring surface motion according to claim 6, 
wherein said step of repeating includes the steps of repeating, starting from 
the level immediately higher than the lowest level to the highest level one by 
one, the steps for each level, including the steps of 

5 regarding, for said first unit image, each of the points of which 

position is estimated for the level immediately lower than said each level as 
a measurement point of said each level, 
^ determining an image segment having a prescribed relation with the 

measurement points of each level, in the subband image corresponding to 
10 said each level, and 
iju determining, in said second unit image, a position of the point 

iy corresponding to each of the measurement points of each level of said first 

nrdt image, by determining the image segment of said subband image of the 
corresponding level obtained for said second unit image, that represents the 
15 highest correlation with the texture of each said image segment. 

8. The method of measuring surface motion according to claim 7, 
S wherein said step for each level further includes the step of estimating a 

position of a point corresponding to each of the measurement points of the 
level immediately above each said level, by interpolating a position of a 
5 point corresponding to each of the measurement points of each said level. 

9. A method of measuring surface motion for measuring motion of 
each portion of an object from a sequence of unit images of the object 
including first and second unit images, comprising the steps of: 

representing the position of each portion of the object surface in said 
5 first unit image by a mesh model including a plurality of nodes of which 
neighboring relations with each other are determined; 

transforming the first and second unit images to a plurality of 
subband images with a predetermined plurality of resolutions; 

generating, from said mesh model, a plurality of coarse mesh models 
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10 having node densities different from each other and lower than the node 
density of said mesh model, said plurahty of coarse mesh models being in 
correspondence with the plurality of subband images; 

obtaining overlay images by overlaying on said plurahty of subband 
images of each of said first and second unit images, corresponding one of the 
15 plurahty of coarse mesh models; and 

calculating cross-correlation of the texture map of the overlay images 
obtained from the subband images corresponding to the first and second unit 
images, finding correspondence between each node of said mesh model of 
said first unit image with each node of the mesh model of said second unit 
:,Q 20 image, and calciolating relative change in position of each node from said 
y first unit image to said second unit image, 

iU 10. The method of measuring surface motion according to claim 9, 

wherein said step of calculating includes the steps of 

calcvdating cross-correlation of the texture map of said overlay 
^ images obtained from first subband images of said first and second unit 

|db 5 images to find correspondence between first said coarse mesh model 

correspondiag to said first subband image of said second unit image with 
Q each node of said first coarse mesh model corresponding to said first 

subband image of said first unit image, so as to calculate relative change in 
position of each node of said first coarse mesh model from said first unit 
10 image to said second unit image, 

deforming a second coarse mesh model having node density higher 
than said first coarse mesh model, of said second unit image in accordance 
with change in position of each node of said first coarse mesh model, said 
second coarse mesh model being in correspondence with a second subband 
15 image having spatial frequency higher than that of said first subband image, 
and 

calculating cross -correlation of the texture map of said overlay 
images obtained from said second subband images of said first and second 
unit images to find correspondence between each node of said second coarse 
20 mesh model corresponding to said first subband image of said second unit 
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image with each node of said second coarse mesh model corresponding to 
said second subband image of said first unit image, thereby calculating 
relative change in position of each node of said second coarse mesh model 
from said first unit image to said second unit image. 

11. The method of measuring surface motion according to claim 10, 
wherein said calculation of cross-correlation is performed using an area 
defined by each node of said coarse mesh model as a center and surrounded 
by the nodes neighboring said center node as a search segment. 

12. A computer readable recording medium storing a program for 
operating a computer to implement a method of measuring surface motion 
for measuring motion of each portion of an object surface from a sequence of 
unit images of the object including first and second unit images, wherein 

said method of measuring surface motion includes the steps of 

representing a position of each portion of said object surface in said 
first unit image by a mesh model including a plurality of nodes of which 
neighboring relations with each other are determined, 

transforming said first and second unit images each to a plurality of 
subband images with a predetermined plurality of resolutions, 

generating, from said mesh model, a plurality of coarse mesh models 
having node densities different from each other and lower than node density 
of said mesh model, said plurahty of coarse mesh models being in 
correspondence with the plurality of subband images respectively, and 

obtaining overlay images by overlaying on said plurahty of subband 
images of each of said first and second unit images, corresponding one of 
said plurahty of coarse mesh models. 

13. The computer readable recording medium according to claim 12, 
wherein said step of calculating includes the steps of 

calculating cross-correlation of the texture map of said overlay 
images obtained from first subband images of said first and second unit 
images to find correspondence between first said coarse mesh model 
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corresponding to said first subband image of said second unit image with 
each node of said first coarse mesh model corresponding to said first 
subband image of said first unit image, so as to calculate relative change in 
position of each node of said first coarse mesh model from said first unit 
image to said second unit image, 

deforming a second coarse mesh model having node density higher 
than said first coarse mesh model, of said second unit image in accordance 
with change in position of each node of said first coarse mesh model, said 
second coarse mesh model being in correspondence with a second subband 
image having spatial frequency higher than that of said first subband image, 
and 

calculating cross-correlation of the texture map of said overlay 
images obtained from said second subband images of said first and second 
unit images to find correspondence between each node of said second coarse 
mesh model corresponding to said first subband image of said second unit 
image with each node of said second coarse mesh model corresponding to 
said second subband image of said first unit image, thereby calculating 
relative change in position of each node of said second coarse mesh model 
from said first unit image to said second unit image. 

14. The computer readable recording medium according to claim 13, 
wherein said calculation of cross-correlation is performed using an area 
defined by each node of said coarse mesh model as a center and surrounded 
by the nodes neighboring said center node as a search segment. 

15. An apparatus for measuring a surface motion for measuring a 
motion of each portion of a surface of an object from a sequence of unit 
images of the object including first and second unit images, comparing: 

means for representing a position of each portion of said object 
surface in said fiirst unit image by a mesh model including a plurality of 
nodes of which neighboring relations with each other are determined; 

means for transforming said first and second unit images each to a 
plurahty of subband images with a predetermined plurahty of resolutions; 
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means for generating, from the mesh model, a plurality of coarse 
mesh models having node densities different from each other and lower than 
node density of the mesh model, said plurahty of coarse mesh models being 
in correspondence with the plurality of subband images, respectively; 

means for obtaining overlay images by overlaying, on said plurality 
of subband images of each of said first and second unit images, 
corresponding one of said plurality of coarse mesh models; and 

means for calculating cross -correlation of a texture map of said 
overlay images obtained from corresponding subband images of said first 
and second unit images to find correspondence between each node of said 
mesh model of said second unit image with each node of said mesh model of 
said first unit image, thereby calculating relative change in position of each 
node from saud fixst unit image to said second unit image. 

16. The apparatus for measuring surface motion according to claim 
15, wherein said means for calculating includes 

mean for calculating cross-correlation of a texture map of said 
overlay images obtained from said first subband images of said first and 
second unit images to find correspondence between each node of said coarse 
mesh model corresponding to said first subband image of said second unit 
image with each node of said first coarse mesh model corresponding to said 
first subband image of said first unit image, thereby calculating relative 
change in position of each node of S£dd first coarse mesh model from said 
first unit image to said second unit image, 

means for deforming a second coarse mesh model having node 
density higher than said first coarse mesh model of said second unit image 
in accordance with the change in position of each node of said first coarse 
mesh model, said second coarse mesh model being in correspondence to a 
second subband image having spatial frequency higher than said first 
subband image; and 

means for calculating cross-correlation of the texture map of said 
overlay images obtained from said second subband images of said first and 
second unit images to find correspondence between each node of said second 
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coarse mesh model corresponding to said first subband image of said second 
unit image with each node of said second coarse mesh model corresponding 
to said second subband image of said first unit image, thereby calculating 
relative change in position of each node of said second coarse mesh model 
from said first unit image to said second unit image. 

17. The apparatus for measuring surface motion according to claim 
16, wherein said calculation of cross-correlation is performed using an area 
defined by each node of said coarse mesh model as a center and surrounded 
by the nodes neighboring said center node as a search segment. 

18. A computer data signal embodied in a carrier wave and 
encoding a plurahty of sequences of instructions, which, when executed by 
one or more processors, cause said one or more processors to configure an 
apparatus for measuring surface motion of an object, said plurality of 
sequences of instructions including sequences of instructions, which, when 
executed by said one or more processors, cause said one or more processors to 
perform the steps of: 

determining a plurality of measurement points in an image of said 
object of a first unit image, said plurality of measurement points having a 
hierarchical structure of a plurality of levels, with the higher level layer 
including measurement points larger in number than the lower level layer; 

determining points in said object image of said second unit image, 
corresponding to the measurement points of said first unit image of the 
lowest level of the hierarchical structure of seiid plurality levels; and 

repeating, starting from a level immediately upper than the lowest 
level to the highest level, determination of the points corresponding to the 
measurement points of the first unit image, of each level, in the image of 
said object of said second unit image, based on the points corresponding to 
said measurement points of said first unit image at an immediately lower 
level of each level in the image of said object of said second unit image. 

19. The computer data signal according to claim 18, wherein 
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said step of determining measurement points includes the steps of 
determining a plurality of reference points of said object image of 

said first unit image, and 

determining, based on the plurality of reference points, arrangement 

of the measurement points having a predetermined distribution. 

20. The computer data signal according to claim 18, wherein 
said step of determining arrangement of the measurement points 

includes the step of mapping a plurality of points arranged at prescribed 
positions on a pre-selected prescribed three-dimensional plane in accordance 
with the three-dimensional shape of the object surface, to an image plane of 
said image, using the plurality of reference points as the reference. 

21. The computer data signal according to claim 20, wherein 
further comprising the step of 

determining the three-dimensional position of said object by 
reverse-projecting, onto a prescribed three-dimensional plane, the points of 
the image of said object in the second unit image corresponding to said 
measurement points of said first unit image, 

22. The computer data signal according to claim 20, wherein 
said step of determining the points corresponding to the 

measurement points includes the steps of 

obtaining subband images by wavelet transform, with a plurality of 
resolutions same in number as said plurality of levels, the object images of 
said first and second unit images, the plurality of resolutions being in 
correspondence with the plurality of levels and resolution corresponding to a 
lower level being selected to be lower; 

determining correspondence between the corresponding subband 
images of the lowest level and the measurement points of said lowest level, 
for said first unit image; 

determining image segments having a prescribed relation with each 
of the measurement points of said lowest level of said subband images of 



-28- 



said lowest level obtained for said first unit image; and 

determining positions of points ia said second unit image, 
corresponding to each of the measurement points of said lowest level in said 
first unit image, by determining an image segment of said subband image of 
said lowest level obtained for said second unit image that represents the 
highest correlation with the texture of each said image segment. 

23. The computer data signal according to claim 21, wherein 
said step of determining the points corresponding to said 

measurement points further includes the step of estimating position of each 
of the measurement points at the level immediately higher than said lowest 
level by interpolating position of the point corresponding to each of the 
measurement points at the lowest level. 

24. The computer data signal according to claim 23, wherein 
sedd step of repeating includes the steps of repeating, starting from 

the level immediately higher than the lowest level to the highest level one by 
one, the steps for each level, including the steps of 

regarding, for said first unit image, each of the points of which 
position is estimated for the level immediately lower than said each level as 
a measurement point of said each level, 

determining an image segment having a prescribed relation with the 
measurement points of each level, in the subband image corresponding to 
said each level, and 

determining, in said second unit image, a position of the point 
corresponding to each of the measurement points of each level of said first 
unit image, by determining the image segment of said subband image of the 
corresponding level obtained for said second unit image, that represents the 
highest correlation with the texture of each said image segment. 

25. The computer data signal according to claim 23, wherein 
said step for each level further includes the step of estimating a 

position of a point corresponding to each of the measurement points of the 
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level immediately above each said level, by interpolating a position of a 
point corresponding to each of the measurement points of each said level. 

26. A computer data signal embodied in a carrier wave and 
encoding a plurahty of sequences of instructions which, when executed by 
one or more processors, cause said one or more processors to configure an 
apparatus for measuring surface motion of an object, said plurahty of 
sequences of instructions including sequences of instructions, which, when 
executed by said one or more processors, cause said one or more processors to 
perform the steps of: 

representing the position of each portion of the object surface in said 
first unit image by a mesh model including a plurahty of nodes of which 
neighboring relations with each other are determined; 

transforming the first and second unit images to a plurahty of 
subband images with a predetermined plurahty of resolutions; 

generating, from said mesh model, a plurahty of coarse mesh models 
having node densities different from each other and lower than the node 
density of said mesh model, said plurahty of coarse mesh models being in 
correspondence with the plurahty of subband images; 

obtaining overlay images by overlaying on said plurahty of subband 
images of each of said first and second unit images, corresponding one of the 
plurahty of coarse mesh models; and 

calcvilating cross-correlation of texture map of the overlay images 
obtained from the subband images corresponding to the first and second unit 
images, finding correspondence between each node of said mesh model of 
said first unit image with each node of the mesh model of said second unit 
image, and calculating relative change in position of each node from said 
fixst unit image to said second unit image. 

27. The computer data signal according to claim 26, wherein 
said step of calculating includes the steps of 

calculating cross-correlation of the texture map of said overlay 
images obtained from first subband images of said first and second unit 
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images to find correspondence between first said coarse mesh model 
corresponding to said first subband image of said second unit image with 
each node of said first coarse mesh model corresponding to said first 
subband image of said fixst unit image, so as to calculate relative change in 
position of each node of said first coarse mesh model from said first unit 
image to said second unit image, 

deforming a second coarse mesh model having node density higher 
than said first coarse mesh model, of said second unit image in accordance 
with change in position of each node of said first coarse mesh model, said 
second coarse mesh model being in correspondence with a second subband 
image having spatigil frequency higher than that of said first subband image, 
and 

calculating cross-correlation of the texture map of said overlay 
images obtained from said second subband images of said first and second 
unit images to find correspondence between each node of said second coarse 
mesh model corresponding to said first subband image of said second unit 
image with each node of said second coarse mesh model corresponding to 
said second subband image of said first unit image, thereby calculating 
relative change in position of each node of said second coarse mesh model 
from said first unit image to said second unit image. 

28. The computer data signal according to claim 27, wherein 
said calculation of cross-correlation is performed using an area 
defined by each node of said coarse mesh model as a center and surrounded 
by the nodes neighboring said center node as a search segment. 



